AI Observability
User Guide — Seeing What the AI Did, and What It Cost
Module: AI Module › Observability | Last updated: July 2026
Contents
- Overview
- Key Concept: Two Logs, Two Jobs
- Finding Your Way Around
- Usage Logs — Activity and Cost
- Audit Logs — Prompts and Responses
- Viewing Original vs. Redacted Content
- Quick Reference
1. Overview
Observability is where you see a record of everything the AI has done — every request it handled, what it cost, and, for chat-style features, exactly what was asked and answered. It's how you keep an eye on activity, keep costs in check, and review what happened when you need to.
The information is split across two logs: Usage Logs, which answer "what did the AI do and what did it cost?", and Audit Logs, which answer "exactly what was asked and answered?".
What you can do here
- See every AI request, with the model used and whether it succeeded.
- Track token use and estimated cost across your activity.
- Review the exact prompts and responses for chat-style features.
- Filter by domain and date to focus on what matters.
- Keep personal information hidden unless you have permission to see it.
WHERE TO FIND IT — Observability lives under Admin → AI → Observability, with two tabs at the top: Usage Logs and Audit Logs.
2. Key Concept: Two Logs, Two Jobs
The two logs record the same requests from two different angles.
| Usage Logs | Audit Logs | |
|---|---|---|
| Answers the question | "What did the AI do, and what did it cost?" | "Exactly what was asked and answered?" |
| Best for | Tracking activity, usage, and cost | Compliance review and troubleshooting |
| Shows | Model, tokens, cost, and status | The full prompt, response, and the data used to answer |
WHY MATCHING ISN'T IN AUDIT LOGS — Record-matching (like Cognitive De-Duplication) appears in Usage Logs but not in Audit Logs. That's expected: matching doesn't involve a written prompt or answer, so there's nothing for the Audit Log to capture. Audit Logs fill up when you use chat-style features and AI suggestions.
3. Finding Your Way Around
Go to Admin → AI → Observability. The two tabs at the top switch between the logs:
- Usage Logs — the activity-and-cost view (covered in section 4).
- Audit Logs — the prompt-and-response view (covered in section 5).
4. Usage Logs — Activity and Cost
The Usage Logs tab lists one row for every AI request. Above the list, a summary shows the totals for whatever is currently on screen: Total Cost, Tokens (in / out), and the number of Requests.
Filters
- Domain — show all domains, or narrow to just one.
- Since Date — show only requests from a chosen date onward. This becomes available once you've picked a specific domain.
What each column shows
| Column | What it shows |
|---|---|
| Date | When the request happened. |
| Domain | Which data domain it belonged to. |
| Request Type | The kind of request — for example, an embedding used for record-matching. |
| Provider | The AI service that handled it. |
| Model | The specific model that was used. |
| Input Tokens | How much text was sent, counted in tokens (roughly, pieces of words). |
| Output Tokens | How much text came back. |
| In Rate / Out Rate ($/MTok) | The price per million tokens used to work out the cost — recorded as it was at the time of the request. |
| Cost (USD) | The estimated cost of that request. |
| Status | Whether the request succeeded or failed. |
FREE AND LOCAL MODELS SHOW $0 — If you're using the built-in local AI, the cost shows as zero. That's correct — local models don't charge per use, so there's nothing to add up.
5. Audit Logs — Prompts and Responses
The Audit Logs tab lists one row for every request that involved a prompt. Like Usage Logs, it has a Domain filter (all domains or one) and a Since Date filter to focus on recent activity.
What each column shows
| Column | What it shows |
|---|---|
| Action | The button that opens the full detail (see below). |
| Date | When the request happened. |
| Domain | Which data domain it belonged to. |
| Request Type | The kind of request — for example, a chat answer or an AI suggestion. |
| Prompt (redacted) | A short preview of the prompt, with personal information hidden. |
| Duration | How long the request took. |
| Client | Which part of OnCoor made the request. |
| PII | Whether the request contained personal information (Yes / No). |
| Status | Whether the request succeeded or failed. |
Opening a request's detail
- Find the request you want in the list (use the Domain and Since Date filters to narrow it down).
- In the Action column, click the View detail icon (the eye).
- The Audit Log Detail opens, showing everything about that single request.
The detail is grouped into sections:
| Section | What it shows |
|---|---|
| Request | The basics — date, domain, who made the request, the model, how long it took, and whether it contained personal information. |
| Prompt | The text that was sent to the AI. |
| Response | The text the AI sent back. |
| Retrieved Context | Any of your data that was pulled in to help answer the request, each with a relevance score. |
| Security Filter / PII Categories | Extra detail on how the request was screened, and what kinds of personal information (if any) were found. |
6. Viewing Original vs. Redacted Content
To protect privacy, the Prompt, Response, and Retrieved Context are shown Redacted by default — any personal information is hidden. Each of these sections has two buttons:
| Button | What it shows |
|---|---|
| Redacted | The safe view, with personal information hidden. This is what everyone sees by default. |
| Original 🔒 | The exact text, including any personal information. The padlock means it's restricted. |
WHO CAN SEE ORIGINALS — Viewing the original, unredacted text is limited to people with the AI Compliance Reviewer role. If you don't have that role, the Original button is unavailable, and hovering over it explains why. Everyone else sees the redacted version. This keeps sensitive information protected while still letting your team review activity.
You can also tell at a glance whether a request involved personal information: the Request section shows a Contains PII marker (Yes or No). "PII" means personally identifiable information — details that could identify a person, such as names or emails.
7. Quick Reference
A fast lookup for the most common actions.
| I want to… | Do this |
|---|---|
| See what the AI did and what it cost | Observability → Usage Logs |
| Check the total cost so far | Read the Total Cost summary at the top of Usage Logs |
| See activity for one domain only | Usage Logs → pick a Domain |
| Narrow to recent activity | Pick a domain, then set a Since Date |
| Read the exact prompt and answer | Observability → Audit Logs → click the View detail (eye) icon |
| Check whether a request contained personal info | Open the Audit Log detail → look at Contains PII |
| View the original (unredacted) text | Requires the AI Compliance Reviewer role → click Original |
| Understand why matching isn't in Audit Logs | Matching has no prompt — find it in Usage Logs instead |
Source: OnCoor AI Module product documentation, written for end users. For the latest screens and options, always refer to the in-app interface.